Method and system for determining location by implication

ABSTRACT

Systems and methods for determining location by implication are described. A responsive environment includes a location determination method that operates in an area that is only partially instrumented with location-sensing devices. Some of the with location-sensing devices sense location ambiguously. For example, a location-sensing device may be deployed at a boundary between two target objects or areas of interest. The location of the target object, as reported by such devices, is considered ambiguous. While the object or person is known to be in a space, it is not clear which specific space. The location of ambiguously located objects can be disambiguated based on changes in the location of other objects. For example, if a document is placed on a shelf in an office, such action strongly implies that someone is in the office. Therefore, if a person is known to potentially be in the office or the outside hallway, the person&#39;s location is changed to be in the office.

CROSS REFERENCE TO RELATED APPLICATIONS

Application Ser. No. 60/521,613, filed Jun. 6, 2004, entitled ResponsiveEnvironment Sensor Systems With Delayed Activation, which isincorporated herein by reference in its entirety. This applicationclaims priority under 35 U.S.C. section 119(e) from Provisional PatentApplication Ser. No. 60/521,747, filed Jun. 29, 2004, entitledResponsive Environment, which is incorporated herein by reference in itsentirety.

BACKGROUND OF INVENTION

The illustrative embodiments described in the present application areuseful in systems including those for use in context aware environmentsand more particularly are useful in systems including those fordetermining location in a context aware environment.

The term responsive environment may be used to describe an environmentthat has computing capability and access to sensing technology data thatallows the environment control to consider its current state or contextand new events that occur that may change the state or context.

Context-aware environments typically utilize location information as animportant type of context. Global Positioning System (GPS) devices areavailable that provide relatively reliable position determinationfunctionality while outdoors. However, GPS signals are typically tooweak to be effective indoors. Accordingly, the determination of indoorposition is more problematic.

Over the last few years, several approaches to solve the indoor positiondetermination problem have been attempted. Each of these traditionalapproaches places a certain amount of burden on the user. For example,in some systems, the user needs to proceed to a location at which theuser presence can be sensed. In other systems, the user must wear aspecial device that interacts with the environment to indicate location.

In traditional indoor positioning systems, the location context isdetermined by one of a few known methods. In the simplest method, anindoor positioning system utilizes a field-of-view approach. With thismethod, objects have a means of identity and a means of communicatingthis identity when entering the field of view of the location-sensingapparatus. One of three traditional categories of systems may beutilized with this simple method. In one type of system, infraredlocation cones may be utilized. In those systems, the objects announcetheir identity using infrared signals. In a second type of system, videoidentification techniques may be used. In those systems, faces or uniquemarkings are identified and associated with the viewed locations. In athird type of system, electro-magnetic field perturbations may be used.In those systems, objects carry tags that emit identification signalswhen present within the field. There, sensed objects are known to bewithin the location covered by the view of the sensing apparatus.

A second and slightly more complex method for determining location isbased on radio frequency technology. In this method, objects carry an RFtransmitter that can announce the identity of the object. The RF signalis detected at multiple RF receivers and the location is determinedbased on signal strengths across these receivers.

A third system for determining locations is based on ultrasonicsignaling. In this method, objects carry a signaling device, oftencalled a beacon. Signal detectors are placed throughout a room, and thelocation of the object is determined by triangulation among thedetectors that receive the signal from the beaconing device. All ofthese indoor position determination methods are described in a paperentitled “Location of Mobile Devices Using Networked Surfaces,” byHoffman and Scott, as presented at the 4rth International Conference onUbiquitous Computing (UbiComp2002).

A fourth system integrating portions of the systems described above isdescribed in a paper entitled “Location Estimation Indoors by Means ofSmall Computing Power Devices, Accelerometers, Magnetic Sensors, and MapKnowledge” by Vildjiounaite et. al., as presented at the FirstInternational Conference on Pervasive Computing in August of 2002. Inthat system, the user wears sensors that measure variables such asdirection and speed. The sensors broadcast that information via RFsignaling. A host computer receives the information and uses it todetermine the location of the wearer. The location can be adjusted usingtraditional field-of-view approaches.

The first two methods described above have several deficiencies. Forexample, the object must be within a field such as an electromagnetic orvisual/audio field. This is problematic for several reasons. First,humans prefer not to be subjected to a local, relatively high-powercontinuous RF field. Second, such fields need to be omnipresent withcomplete area coverage for these systems to work well. Third, if thefields are omni-present, people may perceive privacy concerns. While thethird method described above does not suffer the same disadvantages ofthe first two methods, it does require that objects wear a device thatis currently bulky and expensive.

Accordingly, among other things, the prior art does not provide acontext-aware environment that can adequately determine positioninformation.

SUMMARY OF INVENTION

The illustrative embodiments described herein overcome the disadvantagesof the prior art by providing a method and system for determininglocation by implication.

In one illustrative embodiment, a responsive environment includes alocation determination method that operates in an area that is onlypartially instrumented with location-sensing devices. Some of thelocation-sensing devices sense location ambiguously. For example, alocation-sensing device may be deployed at a boundary between two areasof interest. The location of the target object, as reported by suchdevices, is considered ambiguous. While the object or person is known tobe in a space, it is not clear which specific space. The location ofambiguously located objects can be disambiguated based on changes in thelocation of other objects. For example, if a document is placed on ashelf in an office, such action strongly implies that someone is in theoffice. Therefore, if a person is known to potentially be in the officeor the outside hallway, the person's location is changed to be in theoffice.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic representation of a representative responsiveenvironment according to an illustrative embodiment of the presentapplication.

FIG. 2 is a schematic representation of an illustrative responsiveenvironment and messaging system for determining location by implicationaccording to an illustrative embodiment of the present application.

FIG. 3 is a schematic representation of an illustrative responsiveenvironment for determining location by implication according to anillustrative embodiment of the present application.

FIG. 4 is a flowchart showing a representative method for determininglocation according to an illustrative embodiment of the presentapplication.

FIG. 5 is a flowchart showing a representative method for determininglocation resolution according to an illustrative embodiment of thepresent application.

FIG. 6 is a flowchart showing a representative method for determininglocation certainty according to an illustrative embodiment of thepresent application.

DETAILED DESCRIPTION

Illustrative embodiments describing systems and methods for responsiveenvironments for determining location by implication are described. Theillustrative embodiments are described with reference to particularrepresentative configurations, sensors and data types, but systemsaccording to the present application may use varied and variableconfigurations, sensor types and data and functionality types.

Several disadvantages of traditional indoor positioning systems havebeen described. The illustrative embodiments of the present applicationprovide several advantages over prior systems. The embodiments describeresponsive environments that enable the use of RFID sensors indetermining location, without requiring the presence of the objectwithin the field. Such systems are advantageous due to health concernsand cost concerns, as fewer RFID readers are needed to determinelocation.

The illustrative embodiments of the present application expand upon atraditional field-of-view position determination system. The embodimentsdescribed allow objects, including people, to be sensed usingfield-of-view techniques. Traditional field-of-view systems assign afield to a particular location. For example, if a person is sensed in afield, then it is determined that the person is in the correspondinglocation. Conversely, if a person is not sensed in that field, then itis determined that the person is not in that corresponding location.However, in the embodiments described herein, fields can also beassigned to the boundaries of locations. The specific location of aperson can be determined based on sensing of other objects in fieldswithin one of the spaces sharing the boundary.

For example, an RFID interrogator can be placed at the door of anoffice. As tagged objects pass though the field, their location is knownto be either the office of the hallway. Accordingly, the actual locationof the tagged object is ambiguous. The office contains additional RFIDinterrogators placed in various locations such as on desks, shelves andin bins. As a person uses these spaces that are quipped with RFIDinterrogators, such as by placing a tagged document on the desk, thelocation of the person is updated to show that the person is actually inthe office. The previous location ambiguity is removed by utilizing theimplication that since something is happening in the office, the personmust be in the office. As can be appreciated, the person did notdirectly get sensed in the office.

If two or more people are potentially in the office and the use of otherobjects within the office cannot fully disambiguate among them, thesystem may take one or more of several actions. First, the system canleave the location as equally ambiguous. Second, the system can updatethe location of all people to be more likely in the office. Third, thesystem can update the location of one or some of the people based onprevious knowledge about the object. For example, the system may knowwho last used the document that was sensed in the office. However, asmore information is gathered from activities within the office, thenmore certainty using standard artificial intelligence techniques such asBayesian networks can be gained and applied to imply location of anobject or person. Such notions depart from the typical use of suchdevices that were previously described in which an object is known to beat a location only if it is within the range (view) of the device.

Referring to FIG. 1, an illustrative responsive environment 10 accordingto an illustrative embodiment of the present application is shown. Therepresentative responsive environment has been implemented in a systemknown as Atira that includes a context management infrastructure thatincludes a layered framework of incremental intelligence in the form ofa PAUR pyramid 20 that has four layers each including components thathave similar overall roles. The components pass messages up to the layerabove. However, different components in a particular layer may providespecialized functionality by subscribing to a subset of messages fromthe layer below.

External stimuli are sensed using physical or logical sensors 31, 33, 35and 37. The stimuli enter the pyramid 2 through sensor/triggercomponents 32, 34, 36, 38 that interact directly with the sensors. Thosetriggers typically only publish into the pyramid rather than subscribeto messages. The lowest layer of the pyramid is the P or Perceptionlayer 28 and it includes several perception components 42, 44. Theperception components may subscribe to stimuli events. Similarly, theperception components may publish to the next higher level. ThePerceptors are used to filter the types of external stimuli that areused to build the context.

The next level of the pyramid 20 is the A—Awareness layer 26. Theawareness layer components 52, 54 are known as Monitors. The monitorsmanage the state of active entities that are known in the context suchas document, application or task entities. The monitors 52, 54 managethe overall state of the environment by updating properties associatedwith entities. They determine the occurrence of activities such as aperson carrying a particular document that may also indicate anadditional change in state. They also manage the relationships among theentities.

The next level of the pyramid 20 is the U—Understanding layer 24. Theunderstanding layer components 62, 64 are known as Grokkers. Thegrokkers determine the types of activities that are underway in theenvironment. The grokkers determine if changes in the context merit achange in behavior in the room, and if so, determines the type ofbehavior and initiates it. Grokkers are also utilized to primeapplications.

The final level of the pyramid 20 is the R—Response layer 22. Theresponse layer components 72, 74 are known as Responders. The responderssemantically drive the environment function and prepare and deliver anannouncement that describes the needed behavior. The applications in theenvironment use the announcements to decide if any function is needed.

The responsive environment 10 includes thin client applications thatreside outside of the context infrastructure 30. For example, aninterface browser application 80 may be used to view objects in theenvironment. Additionally, an application launcher client 82 may be usedto launch external applications based upon the context contained in thePAUR pyramid 20. A Notification Manager can be a thin client applicationwith an interactive component that manages the user's attention. Forexample, the thin clients 80, 82 include actuators 86 and 88 that arepart of the thin client systems. The actuators and thin clients maysubscribe to announcements of the system and can also include triggersto create internal stimuli such as an application-entered environment.

The illustrative responsive environment system described utilizes acentral server computing system comprising one or more DELL® servershaving an INTEL® PENTIUM® processor running the WINDOWS® XP operatingsystem. The system is programmed using the JBOSS system and the JavaMessaging System (JMS) provides the publish/subscribe messaging systemused in the responsive environment.

In an illustrative embodiment, physical sensor 31 is a scanner systemthat also includes a computer that interfaces with the sensor component32 using a serial line or TCP/IP interface. The connections among thephysical systems that comprise the logical system 90 include wirelessand wired connections among physical computers running the appropriateapplications, components and frameworks. Sensors 35, 37 are RFID sensorseach including a computer that interfaces with the respective sensorcomponents using a serial line. Sensor 33 may comprise well-knownsensors such as thermometers, pressure sensors, odor sensors, noisesensors, motion sensors, light sensors, passive infrared sensors andother well-known sensors. Additional well-known communications channelsmay also be used. In the illustrative embodiment described, the JBOSSJMS message space is running on one server while the MySQL system is runusing another server to maintain tables used in the RDF system for modeldatabases. Additionally, the PAUR components such as component 42 areall running on a third server. The thin clients 80, 82 and thin clientcomponents 86, 88 are running on separate client machines incommunication with the system 90.

The responsive environment described herein is illustrative and othersystems may also be used. For example, a querying infrastructure couldbe used in place of the notification or publish/subscribe system that isdescribed above. Similarly, the messaging service could be providedacross systems and even across diverse system architectures usingappropriate translators. While INTEL® processor based systems aredescribed using MICROSOFT® WINDOWS systems, other processors andoperating systems such as those available from Sun Microsystems may beutilized.

Referring to FIG. 2, an illustrative responsive environment 200 andmessaging system 210 for determining location by implication accordingto an illustrative embodiment of the present application is shown.

Responsive environments 200 typically include means for managing thestate of the environment, means for sensing events that may change thatstate and means for driving responses to the change in state. In therepresentative embodiment shown, sensing devices include RFID sensors244, 254. The sensor devices are supported by respective host computersystems 240, 250 for communicating stimulus messages along respectivecommunication channels 242, 252.

The sensors 244, 254 are capable of sensing any object that has an RFIDtag affixed to it such as representative tags 246, 256. As can beappreciated, other types of sensors that can identity objects could beused as well. Each deployed sensor 244, 254 is registered within theresponsive environment 200. In this representative example, theregistration information is stored within a model database 230.

Sensor information 234 includes the name and description of the sensor,and a list of spaces over which the field of view of the sensor extendsas noted in the “DeployedForlnfo” property. The spaces noted in thisproperty are also registered in the model. Space information items 232include name and description data for the space, and a list of validtypes that can be present within the space as noted in the“ElementTypes” property. For example, all object types can be presentwithin an office space, but in a representative embodiment, onlydocuments and devices can be present within a cabinet drawer.

When a sensor 244, 254 detects a tag 246, 256, the sensor creates amessage with information declared on the tag, and posts the message tothe message space 210. One or more software components within theresponsive environment that are running on the computing stations 220 ofthe environment can receive these messages.

Referring to FIG. 3, an illustrative responsive environment 300 fordetermining location by implication according to an illustrativeembodiment of the present application is shown.

In the office 300, there are 8 RFID sensors deployed, as shown by theshaded areas 312, 314, 320, 330, 380, 370, 360, and 350. Since the otherareas of the room 300 do not have sensors, it can be appreciated thatmost of the office area is devoid of electromagnetic fields relating tothe sensors. For example, the chair 340 does not include a sensor.Similarly, the desk 310 has only two areas that are covered by sensors,namely the active work area 312 and the queued work area 314. All of thesensors in the office 300, except the Office Boundary sensor 380 aredeployed for a single space. The Office Boundary sensor 380 is deployedfor the office and the external hallway.

The filing cabinet 320 has a sensor, as does the garbage can 330. Thecoat hook 370 has a sensor as does the project shelf 350 and thereference shelf 360. Each of the sensors can provide information orcontext that may be useful in resolving location ambiguity for an objectin the office 300.

Referring to FIGS. 4–6, various stages of a representative process todetermine location by implication are shown. In the illustrative methoddescribed, when a sensor that spans multiple spaces senses an object,the presence of that object is ambiguously noted for each of the coveredspaces. The presence of the object can be resolved based on lateractivity with one of those spaces, even if the object itself is notfurther detected.

Referring to FIG. 4, a representative method 400 for determininglocation according to an illustrative embodiment of the presentapplication is shown.

In step 410, a representative person, George, walks through the OfficeBoundary. In step 420, the Office Boundary sensor information, asregistered in the model, is retrieved. In step 425, the system checks toensure that the sensor is deployed across at least one defined space. Ifnot, the system reports a configuration error in step 490. If the sensoris deployed and associated with at least one defined space, the processproceeds to step 430 to determine the number of spaces that the sensorcovers for that person type.

If the number of spaces covered is exactly 1, the process proceeds downthe less interesting path of location certainty toward step 445. In step445, the process checks that the type of object sensed is valid for thespace associated with the sensor. If it is not, the process ends in step495.

If in step 445 the process determines that the object is a valid type,it proceeds to step 450 to set the persons location to the associatedspace with certainty. The process then proceeds to step 460 to check forthe need for resolution. If George's location was ambiguous, then theambiguous information is removed in step 480. Otherwise, the processends in step 495.

Returning to step 430, if the process determines that the sensor coversmore than one space, the process proceeds to step 435 to determinewhether the object is a valid type for the space. If so, the processproceeds to step 440 to set the ambiguous location of person George ineach of the spaces covered by the sensor with a proportional certaintyof 1 divided by the number of spaces.

Since the representative sensor 380 of this example is deployed at aboundary, its field of view extends across two spaces, as noted in the“DeployedForlnfo” property. Therefore, since the number of spacescovered by the sensor is greater than 1, George is listed as potentiallyin both the office and the external hallway. Accordingly, George'spresence is ambiguous and the certainty of presence in each of the twospaces is initially set for each at 50 percent.

Referring to FIG. 5, a representative method for determining locationresolution 500 according to an illustrative embodiment of the presentapplication is shown. For illustrative purposes, the example describedabove is continued here. If George walks to the garbage can 330 andthrows out a tagged document, the system may be able to imply certaininformation. Since the sensor 330 is deployed for just one space, namelythe garbage can, the document is unambiguously known to be in thegarbage can. As a result, knowledge of this event can be used to resolveambiguous presence in parent containers such as the room. Accordingly,the parent container data is retrieved and checked for any ambiguouspresences.

In step 510, a sensor senses that a piece of paper was tossed in thegarbage can. In step 520, the location of the paper is determined asdescribed above with reference to FIG. 4. In step 530, the processdetermines whether there is a potential for resolving any ambiguouslocation data. If so, the process proceeds to step 540 to retrieve theparent space information. For example, here, the office location datawould be retrieved. The process proceeds to determine whether there areany objects listed as present ambiguously in the parent space of theoffice. If there are objects listed with ambiguous presence in theoffice such as George at 50%, the process proceeds to step 560 toincrease certainty. An illustrative process for increasing certainty isshown below with reference to FIG. 6.

Referring to FIG. 6, a representative method 600 for determininglocation certainty according to an illustrative embodiment of thepresent application is shown. For illustrative purposes, the exampledescribed above is continued here. If ambiguous location references arefound as in the example, then the event of throwing a document away canbe used to increase certainty in one or more of these ambiguouspresences. If only one person might be in the parent container oroffice, then most certainly, that user was the one to throw the documentaway. Accordingly, we are certain that the user is in the parentcontainer or office.

If more than one person is potentially in the space, then moreprocessing is required. For example, the object of the triggering event,in this case the document, may be associated with a particular person.If the document was a private document that was known to be possessedonly by a single person, then it is very likely, though not absolutelynecessary that it was that person who threw it away. Therefore thepresence of that person is changed to Very-Likely present. As can beappreciated, certainty in such an example is not assured. If the paperwas being passed around a lot, and several of the people had previouslypossessed it, then the last possessor is likely the one to throw itaway, but this is far from certain. Therefore the certainty of thatperson is increased some, based on the tuning of the system. Forexample, if 4 people handled the document, the system might decide toincrease the certainty for the relevant person by only 25 percent closerto certainty.

If the document was public and none of the people were known to havepreviously possessed it, then either the certainty of each is increaseda bit or the system can leave the certainty as is currently set. If thecertainty is increased, tuning parameters can be set to maximize theamount of certainty that can result. For example, a limit is placed onthe system such that certainty based on non-associated objects can go upto 70 percent and no further.

In step 610, the system determines if only one object is presentambiguously in the room. If so, the process proceeds to step 615 andsets the presence of that object in the office to 100 percent certainty.

Otherwise, in step 620, the process determines whether the originatingobject (the piece of trash) was equally associated with all of theambiguous objects in the parent container or office. If so, the systemproceeds to step 625 to increase the certainty level by no more than theremaining uncertainty divided by the number of ambiguous objects presentin the parent container.

Otherwise, the process proceeds to step 630 to iterate through each ofthe ambiguous objects in the office. In step 635, the process determineswhether there is a strong association between the piece of trash and oneof the people that might be in the office. If so, it proceeds to step640 to change the ambiguous location data field for that person to verylikely present. If not, the process proceeds to step 645 to increase thecertainty of location for that particular person by the resolutionfactor described above.

For the purpose of clarity, an example describing a single associationhas been illustrated. However, one of skill in the art will be able topractice the invention as described by relaxing the assumption thatthere is a single association. If the assumption were relaxed, theimpact on the resulting derived system would be to the calculation ofthe increase of certainty and how it is shared among those associated.It would not changes the steps described for the simplified case.

The illustrative embodiments described herein provide a method todetermine location, as defined by presence in a space that departs fromtraditional approaches. In at least one embodiment, the system utilizesa space that can be sparsely instrumented with very inexpensivetechnology. It uses the notion of concurrent activity to help resolvelocation ambiguities that may arise from the limitations of suchinstrumentation.

Co-pending, commonly owned U.S. patent application Ser. No. 10/710,293filed on even date herewith, is entitled Responsive Environment SensorSystems With Delayed Activation and is incorporated herein by referencein its entirety.

Co-pending, commonly owned U.S. patent application Ser. No. 10/710,295,filed on even date herewith, is entitled Method and System ForDeployment of Sensors and is incorporated herein by reference in itsentirety.

The present application describes illustrative embodiments of a systemand method for determining location by implication. The embodiments areillustrative and not intended to present an exhaustive list of possibleconfigurations. Where alternative elements are described, they areunderstood to fully describe alternative embodiments without repeatingcommon elements whether or not expressly stated to so relate. Similarly,alternatives described for elements used in more than one embodiment areunderstood to describe alternative embodiments for each of the describedembodiments having that element.

The described embodiments are illustrative and the above description mayindicate to those skilled in the art additional ways in which theprinciples of this invention may be used without departing from thespirit of the invention. Accordingly, the scope of each of the claims isnot to be limited by the particular embodiments described.

1. A method for determining whether an ambiguous location value of afirst object can be further resolved comprising: sensing a second objectpresence in a field of a sensor; determining whether the location of thesecond object is known unambiguously, if the location of the secondobject is known unambiguously, determining whether the second object isassociated with the first object; and further resolving the ambiguouslocation of the first object using the location of the second object. 2.The method of claim 1, further comprising, determining whether thesecond object is uniquely associated with the first object.
 3. Themethod of claim 2, further comprising, if the second object is uniquelyassociated with the first object, unambiguously resolving the locationof the first object by using the location of the second object.
 4. Themethod of claim 3, wherein, the second object is located in a childlocation of a parent location, further comprising, if the second objectis uniquely associated with the first object, unambiguously resolvingthe location of the first object to be the parent location.
 5. Themethod of claim 2, further comprising, if the second object is notuniquely associated with the first object, determining an increase inlocation probability for the first object and further resolving locationof the first object by using the location of the second object and theincrease in location probability.
 6. The method of claim 5, wherein, anartificial intelligence system is used to determine the increase inlocation probability for the first object.
 7. The method of claim 6,wherein, the artificial intelligence system is a baysian network.
 8. Themethod of claim 1, further comprising, determining whether the secondobject is strongly associated with the first object.
 9. The method ofclaim 8, further comprising, if the second object is strongly associatedwith the first object, ambiguously resolving the location of the firstobject to a very likely probability by using the location of the secondobject.
 10. A method for determining whether an ambiguous location valueof a first object can be further resolved comprising: sensing a secondobject presence in a field of a sensor; determining whether the locationof the second object is known ambiguously, if the location of the secondobject is known ambiguously, determining whether the second object isassociated with the first object; determining whether the ambiguouslocation of the second object provides information to further resolvethe location of the first object; and if the ambiguous location of thesecond object provides information to further resolve the location ofthe first object, further resolving the ambiguous location of the firstobject using the location of the second object.
 11. The method of claim10, further comprising, determining whether the second object isuniquely associated with the first object.
 12. The method of claim 11,further comprising, if the second object is uniquely associated with thefirst object, further resolving the location of the first object byusing the location of the second object.
 13. The method of claim 12,wherein, the second object is ambiguously located in one of a pluralityof child locations of a parent location, further comprising, if thesecond object is uniquely associated with the first object,unambiguously resolving the location of the first object to be theparent location.
 14. The method of claim 11, further comprising, if thesecond object is not uniquely associated with the first object,determining an increase in location probability for the first object andfurther resolving location of the first object by using the location ofthe second object and the increase in location probability.
 15. Themethod of claim 14, wherein, an artificial intelligence system is usedto determine the increase in location probability for the first object.16. The method of claim 15, wherein, the artificial intelligence systemis a baysian network.
 17. The method of claim 10, further comprising,determining whether the second object is strongly associated with thefirst object.
 18. The method of claim 17, further comprising, if thesecond object Is strongly associated with the first object, ambiguouslyresolving the location of the first object by using the ambiguouslocation of the second object and the strength of the association.